K-D Decision Tree: An Accelerated and Memory Efficient Nearest Neighbor Classifier

This paper presents a novel Nearest Neighbor (NN) classifier. NN classification is a well studied method for pattern classification having the following properties; * it performs maximum-margin classification and achieves less than the twice of ideal Bayesian error, * it does not require the knowledge on pattern distributions, kernel functions or base classifiers, and * it can naturally be applied to multiclass classification problems. The drawbacks are A) inefficient memory use, B) ineffective pattern classification speed, and so on. This paper deals with the problems A and B. In most of the cases, NN search algorithms, such as k-d tree, are employed as a pattern search engine of the NN classifier. However, NN classification does not always require the NN search. Based on this idea, we propose a novel algorithm named k-d decision tree (KDDT). Since KDDT requires Voronoi condensed prototypes, it is less memory consuming than naive NN classifiers. We have confirmed that KDDT is much faster than NN search based classifier through the comparative experiment (from 9 to 369 times faster than NN search based classifier). Keyword nearest neighbor classifier, k-d tree, Voronoi condensing, local nearest neighbor search, safe node merging